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Abstract 

Background: There has been much debate about the appropriate statistical methodology for the evaluation of 
malaria field studies and the challenges in interpreting data arising from these trials. 

Methods: The present paper describes, for a pivotal phase III efficacy of the RTS, S/AS01 malaria vaccine, the 
methods of the statistical analysis and the rationale for their selection. The methods used to estimate efficacy of 
the primary course of vaccination, and of a booster dose, in preventing clinical episodes of uncomplicated and 
severe malaria, and to determine the duration of protection, are described. The interpretation of various measures 
of efficacy in terms of the potential public health impact of the vaccine is discussed. 

Conclusions: The methodology selected to analyse the clinical trial must be scientifically sound, acceptable to 
regulatory authorities and meaningful to those responsible for malaria control and public health policy. 

Trial registration: Clinicaltrials.gov NCT00866619 



Background 

In May 2009, the first large scale, phase III efficacy trial 
assessing a candidate malaria vaccine was launched across 
seven countries in sub-Saharan Africa. The vaccine candi- 
date RTS, S/AS01 targets the pre-erythrocytic stage of the 
Plasmodium falciparum parasite. This phase III trial is 
designed to determine the efficacy of the vaccine against 
clinical malaria in young children to support the file sub- 
mission for regulatory review by European and African 
regulatory authorities. Additionally, a broad range of sec- 
ondary efficacy endpoints are included to evaluate the full 
potential direct impact of the vaccine on child health. 
These include severe malaria and malaria-specific mortal- 
ity as well as all-cause hospitalization and mortality. 

Malaria is a common disease in sub-Saharan children 
and so it may at first appear relatively simple to design 
trials to demonstrate the effects of interventions with rea- 
sonable sample sizes. However, the design of such trials 
comes with specific statistical challenges. All individuals 



* Correspondence: marc.lievens@gskbio.com 
1 GlaxoSmithKline Biologicals, Wavre, Belgium 
Full list of author information is available at the end of the article 



are not at the same level or risk of malaria disease; trans- 
mission varies widely across trial sites. Malaria control 
measures are being scaled-up across sub-Saharan Africa, 
but at differing rates and coverage. Statistical methods 
must be able to adjust for the confounding effect of 
explanatory variables and take into account the fact that 
multiple events within the same individual are not 
independent. 

Phase II data suggest that while RTS, S/AS01 offers 
approximately 50% protection against clinical malaria 
disease, the likely mechanism of protection is reduction 
in the risk of disease associated with an infectious bite in 
all individuals rather than complete protection in 50% of 
individuals. A vaccine with this kind of effect is often 
referred to as a "leaky vaccine". A consequence of the use 
of a leaky malaria vaccine is that over time both vacci- 
nated and comparator children will continue to experi- 
ence malaria episodes and that in high transmission areas 
following sufficient exposure and extended follow-up, all 
individuals will experience an infection, regardless of 
whether they were vaccinated or not. However, this does 
not mean that vaccination would not provide public 
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health benefit, since the number of episodes could be sig- 
nificantly reduced as well as the risk of complications. 

In endemic areas, young children may experience sev- 
eral episodes of malaria per year and gradually acquire 
immunity against malaria disease through repeated expo- 
sure to parasites. Naturally acquired immunity is predo- 
minantly directed to the blood stage of the infection, in 
contrast to the vaccine, which induces protection 
through an effect on the pre-erythrocytic stage. Vaccine 
trials are conducted superimposed on this background 
making long-term vaccine efficacy difficult to disentangle 
from protection acquired by exposure to infection. 

Vaccine efficacy is calculated as VE = 1 - R where R may 
be a ratio of risks, rates or hazards. All of these methods 
have been used to license vaccines. The choice of method 
is dependent upon the data, but it is also important that 
the output is meaningful to the end user: those making 
vaccine policy decisions and eventually parents. In this 
trial, vaccine efficacy is calculated using comparison of 
risks, rates, or hazards depending on the endpoint. The 
purpose of this paper is to set out the statistical methods 
that will be used to estimate the efficacy of RTS, S/AS01. 
The choice of statistical analysis for endpoints in the RTS, 
S/AS01 phase III trial is used to illustrate the advantages 
and disadvantages of each method. 

Trial design 

The RTS, S/AS01 pivotal phase III trial is a double-blind 
(observer-blind), randomized, controlled study conducted 
in 11 centres located across seven countries in sub- 
Saharan Africa enrolling children in two age categories: 
6-12 weeks and 5-17 months old with a minimum of 
6,000 subjects in both age categories up to a maximum 
of 16,000 children and infants overall. In both age cate- 
gories, the subjects are randomized to one of the three 
study groups in a 1:1:1 ratio; (i) RTS/SAS01 given in a 
0,1,2-month schedule + a RTS, S/AS01 booster dose at 
Month 20 (group R3R); (ii) RTS/SAS01 given in a 0,1,2- 
month schedule + a booster dose of control vaccine at 
Month 20 (group R3C); (iii) control vaccine given in a 
0,1,2-month schedule + a booster dose of control vaccine 
at Month 20 (group C3C) [1]. Children in the 6-12 week 
age group receive the primary course of RTS, S/AS01 co- 
administered with their routine infant immunizations 
(diphtheria, tetanus, whole cell pertussis, hepatitis B, 
Haemophilus influenzae type B and oral polio vaccines). 

The full list of efficacy endpoints is given in Table 1. 
Case definitions are given in the companion papers 
including those for clinical malaria and other morbidity 
endpoints [1] and severe malaria [2]. 

The co-primary endpoints are efficacy against clinical 
malaria for 12 months post Dose 3 in both the 6-12 
weeks and 5-17 months age groups. The analysis of effi- 
cacy will be carried out when 6,000 children in an age 



category have been enrolled and followed up for 
12 months post Dose 3. Vaccine efficacy (VE) against 
first or only episodes will be determined using a standard 
formula: VE = 100 x (1 - HR)% where HR is the hazard 
ratio estimated from Cox regression model. Following an 
episode of malaria, an individual will be removed from 
the risk set and subjects who are lost to follow-up, who 
are withdrawn, or who die will be included up to the date 
of loss, withdrawal or death [3]. Sites are considered as 
strata, thus allowing for the fact that the temporal pattern 
of incidence rate in the control group may vary from site 
to site. The time scale for analysis is the elapsed time 
since 14 days after dose 3 (for the ATP analysis), and 
elapsed time after dose 1 (for the ITT analysis), rather 
than calendar time. This allows all randomized subjects 
to be included in the risk set at the start of follow-up, 
and it simplifies analysis of duration of protection. 
A common case definition will be applied in all sites, 
chosen to have high specificity in all sites. 

The primary analysis will be stratified for study site and 
unadjusted for other covariates; however, vaccine efficacy 
estimates adjusted for covariates will also be presented. As 
there are two co-primary endpoints, 97.5% confidence 
intervals (CIs) on VE will be calculated, ensuring an over- 
all 2-sided 5% alpha-level. Therefore, p-values lower than 
0.025 will be considered significant for the primary end- 
points. Assuming at least 5400 evaluable subjects (rando- 
mized 2:1), an attack rate in controls of 10/100 per child 
year at risk (cyr) over the follow-up period from 2 weeks 
post Dose 3 to 1 year post Dose 3 and a true vaccine effi- 
cacy of 30%, the study has 90% power to detect a lower 
limit of the 97.5% CI around estimated VE above 0%. All 
secondary endpoints will be evaluated using 95% CIs. 

VE against all episodes will be calculated by an inci- 
dence rate ratio (IRR) using a negative binomial regres- 
sion model controlling for interdependence between 
episodes within the same subject. The 95% CI and 
p-values on VE estimates, defined as 100 x (1 - IRR)%, 
will be calculated from this model. Events occurring 
within 14 days following a malaria episode meeting the 
malaria case definition under evaluation will not contri- 
bute, to minimize counting treatment failures as an epi- 
sode resulting from a new infection and the total time at 
risk will be adjusted. 

VE against severe malaria will be evaluated when 250 
subjects have reported an episode of severe malaria meet- 
ing the primary case definition. Recruitment of 250 
affected subjects provides 90% power to detect 30% VE 
with a lower limit of 95% CI above 0%. The primary analy- 
sis will compare the pooled RTS, S/AS01 groups versus 
the control group (R3R + R3C vs C3C). VE against severe 
malaria will be estimated as 1 - RR where RR is the risk 
ratio (proportion of subjects reporting severe malaria in 
the RTS, S/AS01 group over the proportion in the control 
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Table 1 Efficacy endpoints 

Primary Efficacy Efficacy against clinical malaria when primary immunization starts at 6-12 weeks, or 5-17 months of age 

Endpoint Occurrence of cases of malaria meeting the primary case definition for clinical malaria over a period starting 14 

days post Dose 3 for 12 months in children aged 6-12 weeks 

Occurrence of cases of malaria meeting the primary case definition for clinical malaria over a period starting 14 days 
post Dose 3 for 12 months in children aged 5-17 

Secondary Efficacy Efficacy against severe disease 

Endpoints Occurrence of severe malaria meeting the primary and secondary case definitions 

Efficacy against incident severe anaemia and malaria hospitalization 

Occurrence of incident severe anaemia and malaria hospitalization meeting the primary and secondary case definitions 
Duration of efficacy of a primary course 

For a primary schedule without a boost, the occurrence of clinical malaria meeting the primary case definition analysed 
over the time periods starting 14 days post Dose 3 until boost, boost until study end and 14 days post Dose 3 until 
study end 

Potential added benefit of a booster dose 

For a primary schedule with and without a boost, the occurrence of clinical malaria meeting the primary case definition 
analysed over the time period starting at boost until study end 

Efficacy under different transmission settings 

For each site, occurrence of clinical malaria disease meeting the primary case definitions 

Efficacy against secondary case definitions of clinical malaria 

Occurrence of clinical malaria disease meeting the secondary case definitions 

Efficacy against prevalence of parasitaemia 

Presence of parasitaemia at 18 months and 30 months post Dose 3 and 12 months after the booster dose 
Efficacy against prevalence of moderate and severe anaemia 

Presence of moderate and severe anaemia at 18 months and 30 months post Dose 3 and 12 months after the booster 
dose 

Efficacy against other serious illness 

Occurrence of other serious illness meeting the primary and secondary case definitions. Other serious illness is all 
medical hospitalization, sepsis and pneumonia 

Efficacy against fatal malaria and all-cause mortality 

Occurrence of fatal malaria (meeting the case definitions) and all-cause mortality 

Effect on growth 

Compare the height/length, weight and mid-upper arm circumference for age z-score 
Gender-specific efficacy 

In male and female children, the occurrence of clinical malaria disease meeting the primary case definition 



group) together with 95% CIs. Analysis of all events will be 
performed in a similar way as for clinical malaria disease. 
Table 2 summarizes the analytic approach of efficacy 
applied to each endpoint. 

The benefit of the booster dose will be evaluated by 
the comparison of RTS, S/AS01 recipients receiving a 
booster dose compared to control booster (R3R vs R3C) 
over the 12 months following booster dose. Children 
receiving a primary schedule with or without booster 
dose will be compared to controls as well (R3R vs C3C 
and R3C vs C3C). 

At 18 and 30 months post primary immunization, a 
cross sectional survey will sample all participating chil- 
dren to assess vaccine efficacy against prevalent parasi- 
taemia and anaemia. This will be estimated by the 
prevalence ratio with 95%CI. 

The primary efficacy analysis will be performed on the 
according to protocol (ATP) population over the period 
starting 14 days post Dose 3. The ATP population 
includes all subjects meeting the inclusion criteria who 
received all vaccinations according to protocol and 



Table 2 



Endpoint 



Risk ratio 



Comparison of proportion 
affected 



At least one episode: 
Severe malaria 

Incident severe anaemia 

Malaria mortality 

All cause mortality 

Prevalent P. folciporum 
infection 

Prevalent anaemia 



Rate ratio 



Negative binomial regression 
model 



All episodes: 

Clinical malaria 

Severe malaria 
Malaria hospitalization 
All cause hospitalization 
Sepsis 
Pneumonia 



Hazard Ratio 

Cox regression model 



First or only episode: 
Clinical malaria 
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contribute to time at risk starting 14 days post dose 3. 
The choice to perform the primary analysis on the ATP 
population reflects the thought that the main outcome is 
to describe vaccine efficacy for a vaccinated population. 
However, analyses will also be conducted on a modified 
intention to treat (ITT) population, which includes all 
subjects who received at least one dose of vaccine. The 
ITT population will be analysed by randomization assign- 
ment and start time at risk from the day of first vaccina- 
tion. Subjects randomized but not vaccinated are not 
under efficacy surveillance. 

Assessment of vaccine efficacy against clinical 
malaria 

Case definitions - malaria is characterized by fever caused 
by parasitaemia. In malaria endemic regions it is not 
uncommon for children to carry asymptomatic infections 
with P. falciparum. Fever in these children may be caused 
by disease other than malaria. In order to improve speci- 
ficity, cases in this trial are defined as children who are 
unwell and have presented to the clinic with fever and 
parasite density above a threshold value [4]. A common 
parasite density threshold (>5,000/mcl) was chosen to 
have good specificity in all sites, recognizing that this 
means sensitivity will vary from site to site. Secondary 
case definitions include other thresholds [1]. 

The percentage of children free of malaria by the end of 
the trial can be compared but efficacy will depend on the 
level of incidence and the duration of the trial so this mea- 
sure of efficacy is difficult to compare between trials and it 
does not reflect the reduction in disease burden due to 
vaccination. As a primary analysis, vaccine efficacy is esti- 
mated by evaluating the hazard ratio of first or only epi- 
sodes, allowing evaluation of the direct effect of the 
vaccine. This is in line with the hypothesized mechanism 
of RTS, S/AS vaccination, namely that rather than provid- 
ing complete protection to half of the individuals (all or 
nothing type vaccine), it protects by reducing the risk of 
disease following an infectious bite by a certain proportion 
for all individuals. The Cox model allows for different 
levels of baseline risk across study sites and can include 
baseline covariates. Under the assumption that vaccine 
efficacy is not different across study sites and the hazard 
ratio is constant over time, it is a good method to reduce 
variability and control for confounding, particularly as 
data for the primary endpoint will be pooled across multi- 
ple transmission settings. Also, analysis on vaccine efficacy 
by study site is a secondary analyses planned for by the 
protocol. This methodology is well established and has 
been broadly used to evaluate previous malaria interven- 
tions [5-13] and to license many drugs and vaccines. This 
method is acceptable to regulatory authorities and WHO 
as a measure of VE [1,14], although a shortcoming of the 



vaccine efficacy calculated from hazard ratios could be 
that it is not intuitively understood. 

Vaccine efficacy is also calculated against all episodes of 
clinical malaria. The methodology used to do this must 
take into account the fact that the risk in all individuals is 
not equal; some individuals are more susceptible than 
others related to host factors such as genetic background, 
health and behaviour or to the level of exposure to bites 
by infectious mosquitoes in the community. Moreover, the 
risk of malaria may not be constant over subsequent epi- 
sodes. Vaccine efficacy is calculated by comparing inci- 
dence rates using a negative binomial regression model 
allowing for individual heterogeneity arising from non- 
independence of multiple episodes within the same sub- 
ject. This model accounts for heterogeneity among indivi- 
duals and considers non-independent multiple episodes 
[15,16]. Vaccine efficacy based on a comparison of inci- 
dence rates is perceived as a straightforward measure that 
is meaningful to policy makers and can be used for the 
evaluation of the economic consequences of vaccine 
introduction. 

Assessment of vaccine efficacy against severe 
malaria 

Unlike uncomplicated clinical malaria, which is a rela- 
tively common disease from which patients usually 
recover completely with treatment, severe malaria is a 
rare event associated with a high case fatality rate even 
in the presence of optimal treatment and repeated epi- 
sodes in the same individual are uncommon [17]. As 
such, severe malaria remains an important cause of 
childhood mortality in sub-Saharan Africa [18,19]. 

Vaccine efficacy against severe malaria disease will be 
analysed as a risk ratio, comparing the proportion of chil- 
dren affected by at least one episode of severe malaria in 
each group. The key advantage of this measure is that it is 
readily interpretable; the vaccine reduces the risk of severe 
disease over the specified time frame by a proportion. This 
methodology is also used for each of the endpoints, which 
share the same features of being rare and life-threatening 
(Table 2). 

In a secondary analysis, all episodes of severe malaria 
will be investigated in the same way as all episodes of 
clinical malaria. 

Assessment of vaccine protection over time 

The assessment of the evolution of vaccine efficacy against 
clinical malaria over time is the most challenging aspect of 
the analysis; it is not a limitation of the statistical metho- 
dology but of the data set itself. Two analytic methods will 
be used to evaluate the evolution of protection against 
clinical malaria over the 30-month follow up period. The 
first will compare the study groups R3C and C3C from the 
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beginning to the end of the study and the second will esti- 
mate vaccine efficacy over shorter time periods; from 14 
days post dose 3 to 17 months and from 18 months to 30 
months. To evaluate duration of protection, analyses of all 
events is more informative than first or only episodes. 

The challenge lies in the interpretation of the results 
that these analyses will provide because neither methodol- 
ogy is able to disentangle the effects of vaccine induced 
immunity and naturally acquired protection. In most parts 
of Africa, children acquire a high level of protection 
against the severe forms of malaria in early childhood as a 
result of repeated exposure to blood stage parasites. It is 
expected that the vaccinees also acquire immunity as a 
result of exposure to infection. The difficulty is that there 
is no immunological assay that can tell whether vaccinees 
have acquired natural immunity at the same rate as con- 
trols and allow adjustment for this in the analysis. If nat- 
ural immunity is acquired at a similar rate in controls and 
vaccines, then estimates of vaccine efficacy will approxi- 
mate true vaccine efficacy, but if vaccinees and controls 
acquire natural immunity at a different rate, estimates will 
not reflect true vaccine efficacy. Although efficacy esti- 
mates based on first or only episodes may not be very 
informative of duration of protection, the Cox propor- 
tional hazards method allows to investigate whether the 
proportionality of hazards remains constant over time (i.e. 
that observed vaccine effect remains constant) by examin- 
ing cumulative incidence plots, Schoenfeld residuals and 
models with time-dependent covariates. If there is no sup- 
port from the data that the HR changes over time, one 
could assume a constant effect over the follow-up. How- 
ever, the opposite is not true: violating the proportionality 
of hazards assumption does not necessarily mean waning 
efficacy. Violating a proportionality of hazards assumption 
in the presence of declining attack rates may suggest 
decreasing susceptibility or heterogeneity in exposure 
rather than waning vaccine efficacy. The approach of 
breaking the surveillance period into shorter time periods, 
sometimes described as "resetting the clock", is intuitive. 
However, the interpretation remains complicated by the 
fact that the control group is no longer directly compar- 
able in the second time period due to different exposure 
in the first time period and the concomitant development 
of natural immunity. 

As relevant to policy makers as the evolution of vaccine 
efficacy over time, is the presentation of results addressing 
the fundamental hypothesis underlying the development 
strategy for RTS, S/AS01. The hypothesis is that a partially 
efficacious vaccine would be of great value in endemic 
areas, if it was able to be given to young children and pro- 
vide them with protection against the severe life-threaten- 
ing forms of the disease until they had acquired natural 
protection. This can be assessed by presenting an analysis 
of the number of cases averted by the vaccine over time. 



Covariates 

A covariate can be defined as a qualitative factor or a 
quantitative variable that is expected to have an influ- 
ence on study endpoint(s) to be analysed. Covariates 
can be highly variable and may include disease charac- 
teristics, social or economic factors and geographic or 
demographic data [20]. 

Study centres participating in this trial represent a range 
of malaria transmission intensities across sub-Saharan 
Africa, including the intense seasonal transmission of 
West Africa and perennial transmission in East Africa. 
Other factors might influence the pattern of malaria dis- 
ease in a population and may be taken into account as 
covariates for data analyses. The importance to controlling 
for heterogeneity in malaria risk to avoid underestimating 
vaccine efficacy has been highlighted earlier [21]. 

All the covariates collected in this trial have known 
associations with malaria risk. Cox regression allows for 
seasonal variation of malaria incidence and adding study 
centres as a covariate allows for different levels of base- 
line risk and different patterns of seasonality. The trial 
collected a number of potential covariates including age, 
bed net use [22-24], distance from nearest inpatient/out- 
patient facility [25], pneumococcal vaccination status 
[26,27], ethnicity [28,29] and nutritional status [30,31]. 

Conclusion 

In conclusion, the analytic methods selected to analyse 
the results of the RTS, S/AS01 phase III clinical trial 
must be scientifically sound, acceptable to regulatory 
authorities and meaningful to those responsible for 
malaria control and public health policy. These needs 
have been taken into account and resulted in a list of pri- 
mary and secondary endpoints, each with their appropri- 
ate analytic methods. Finally, an understanding of the 
biological action and full benefit/risk profile of the vac- 
cine will only be gained by looking at the full picture that 
is painted by the entire range of predefined endpoints 
and planned analyses. 
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